A Real-time Calculus Approach for Integrating Sporadic Events in Time-triggered Systems

In time-triggered systems, where the schedule table is predefined and statically configured at design time, sporadic event-triggered (ET) tasks can only be handled within specially dedicated slots or when time-triggered (TT) tasks finish their execution early. We introduce a new paradigm for synthesizing TT schedules that guarantee the correct temporal behavior of TT tasks and the schedulability of sporadic ET tasks with arbitrary deadlines. The approach first expresses a constraint for the TT task schedule in the form of a maximal affine envelope that guarantees that as long as the schedule generation respects this envelope, all sporadic ET tasks meet their deadline. The second step consists of modeling this envelope as a burst limiting constraint and building the TT schedule via simulating a modified Least-Laxity-First (LLF) scheduler. Using this novel technique, we show that we achieve equal or better schedulability and a faster schedule generation for most use-cases compared to other approaches inspired by, e.g., hierarchical scheduling. Moreover, we present an extension to our method that finds the most favourable schedule for TT tasks with respect to ET schedulability, thus increasing the probability of the computed TT schedule remaining feasible when ET tasks are later added or changed

The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning

No free lunch theorems for supervised learning state that no learner can solve all problems or that all learners achieve exactly the same accuracy on average over a uniform distribution on learning problems. Accordingly, these theorems are often referenced in support of the notion that individual problems require specially tailored inductive biases. While virtually all uniformly sampled datasets have high complexity, real-world problems disproportionately generate low-complexity data, and we argue that neural network models share this same preference, formalized using Kolmogorov complexity. Notably, we show that architectures designed for a particular domain, such as computer vision, can compress datasets on a variety of seemingly unrelated domains. Our experiments show that pre-trained and even randomly initialized language models prefer to generate low-complexity sequences. Whereas no free lunch theorems seemingly indicate that individual problems require specialized learners, we explain how tasks that often require human intervention such as picking an appropriately sized model when labeled data is scarce or plentiful can be automated into a single learning algorithm. These observations justify the trend in deep learning of unifying seemingly disparate problems with an increasingly small set of machine learning models

CoLA: Exploiting Compositional Structure for Automatic and Efficient Numerical Linear Algebra

Many areas of machine learning and science involve large linear algebra problems, such as eigendecompositions, solving linear systems, computing matrix exponentials, and trace estimation. The matrices involved often have Kronecker, convolutional, block diagonal, sum, or product structure. In this paper, we propose a simple but general framework for large-scale linear algebra problems in machine learning, named CoLA (Compositional Linear Algebra). By combining a linear operator abstraction with compositional dispatch rules, CoLA automatically constructs memory and runtime efficient numerical algorithms. Moreover, CoLA provides memory efficient automatic differentiation, low precision computation, and GPU acceleration in both JAX and PyTorch, while also accommodating new objects, operations, and rules in downstream packages via multiple dispatch. CoLA can accelerate many algebraic operations, while making it easy to prototype matrix structures and algorithms, providing an appealing drop-in tool for virtually any computational effort that requires linear algebra. We showcase its efficacy across a broad range of applications, including partial differential equations, Gaussian processes, equivariant model construction, and unsupervised learning.Comment: Code available at https://github.com/wilson-labs/col

A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks

Unlike conventional grid and mesh based methods for solving partial differential equations (PDEs), neural networks have the potential to break the curse of dimensionality, providing approximate solutions to problems where using classical solvers is difficult or impossible. While global minimization of the PDE residual over the network parameters works well for boundary value problems, catastrophic forgetting impairs the applicability of this approach to initial value problems (IVPs). In an alternative local-in-time approach, the optimization problem can be converted into an ordinary differential equation (ODE) on the network parameters and the solution propagated forward in time; however, we demonstrate that current methods based on this approach suffer from two key issues. First, following the ODE produces an uncontrolled growth in the conditioning of the problem, ultimately leading to unacceptably large numerical errors. Second, as the ODE methods scale cubically with the number of model parameters, they are restricted to small neural networks, significantly limiting their ability to represent intricate PDE initial conditions and solutions. Building on these insights, we develop Neural IVP, an ODE based IVP solver which prevents the network from getting ill-conditioned and runs in time linear in the number of parameters, enabling us to evolve the dynamics of challenging PDEs with neural networks.Comment: ICLR 2023. Code available at https://github.com/mfinzi/neural-iv

PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization

While there has been progress in developing non-vacuous generalization bounds for deep neural networks, these bounds tend to be uninformative about why deep learning works. In this paper, we develop a compression approach based on quantizing neural network parameters in a linear subspace, profoundly improving on previous results to provide state-of-the-art generalization bounds on a variety of tasks, including transfer learning. We use these tight bounds to better understand the role of model size, equivariance, and the implicit biases of optimization, for generalization in deep learning. Notably, we find large models can be compressed to a much greater extent than previously known, encapsulating Occam's razor. We also argue for data-independent bounds in explaining generalization.Comment: NeurIPS 2022. Code is available at https://github.com/activatedgeek/tight-pac-baye

Carbon budget of the Harvard Forest Long- Term Ecological Research site: pattern, process, and response to global change

How, where, and why carbon (C) moves into and out of an ecosystem through time are long- standing questions in biogeochemistry. Here, we bring together hundreds of thousands of C- cycle observations at the Harvard Forest in central Massachusetts, USA, a mid- latitude landscape dominated by 80- 120- yr- old closed- canopy forests. These data answered four questions: (1) where and how much C is presently stored in dominant forest types; (2) what are current rates of C accrual and loss; (3) what biotic and abiotic factors contribute to variability in these rates; and (4) how has climate change affected the forest- s C cycle? Harvard Forest is an active C sink resulting from forest regrowth following land abandonment. Soil and tree biomass comprise nearly equal portions of existing C stocks. Net primary production (NPP) averaged 680- 750 g C·m- 2·yr- 1; belowground NPP contributed 38- 47% of the total, but with large uncertainty. Mineral soil C measured in the same inventory plots in 1992 and 2013 was too heterogeneous to detect change in soil- C pools; however, radiocarbon data suggest a small but persistent sink of 10- 30 g C·m- 2·yr- 1. Net ecosystem production (NEP) in hardwood stands averaged ~300 g C·m- 2·yr- 1. NEP in hemlock- dominated forests averaged ~450 g C·m- 2·yr- 1 until infestation by the hemlock woolly adelgid turned these stands into a net C source. Since 2000, NPP has increased by 26%. For the period 1992- 2015, NEP increased 93%. The increase in mean annual temperature and growing season length alone accounted for ~30% of the increase in productivity. Interannual variations in GPP and NEP were also correlated with increases in red oak biomass, forest leaf area, and canopy- scale light- use efficiency. Compared to long- term global change experiments at the Harvard Forest, the C sink in regrowing biomass equaled or exceeded C cycle modifications imposed by soil warming, N saturation, and hemlock removal. Results of this synthesis and comparison to simulation models suggest that forests across the region are likely to accrue C for decades to come but may be disrupted if the frequency or severity of biotic and abiotic disturbances increases.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/163495/3/ecm1423_am.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/163495/2/ecm1423-sup-0001-AppendixS1.pdfhttp://deepblue.lib.umich.edu/bitstream/2027.42/163495/1/ecm1423.pd

Altered Insulin Receptor Signalling and β-Cell Cycle Dynamics in Type 2 Diabetes Mellitus

Insulin resistance, reduced β-cell mass, and hyperglucagonemia are consistent features in type 2 diabetes mellitus (T2DM). We used pancreas and islets from humans with T2DM to examine the regulation of insulin signaling and cell-cycle control of islet cells. We observed reduced β-cell mass and increased α-cell mass in the Type 2 diabetic pancreas. Confocal microscopy, real-time PCR and western blotting analyses revealed increased expression of PCNA and down-regulation of p27-Kip1 and altered expression of insulin receptors, insulin receptor substrate-2 and phosphorylated BAD. To investigate the mechanisms underlying these findings, we examined a mouse model of insulin resistance in β-cells – which also exhibits reduced β-cell mass, the β-cell-specific insulin receptor knockout (βIRKO). Freshly isolated islets and β-cell lines derived from βIRKO mice exhibited poor cell-cycle progression, nuclear restriction of FoxO1 and reduced expression of cell-cycle proteins favoring growth arrest. Re-expression of insulin receptors in βIRKO β-cells reversed the defects and promoted cell cycle progression and proliferation implying a role for insulin-signaling in β-cell growth. These data provide evidence that human β- and α-cells can enter the cell-cycle, but proliferation of β-cells in T2DM fails due to G1-to-S phase arrest secondary to defective insulin signaling. Activation of insulin signaling, FoxO1 and proteins in β-cell-cycle progression are attractive therapeutic targets to enhance β-cell regeneration in the treatment of T2DM